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The  purpose  of  this  brief  note  Is  to  show  that  we  can  obtain 

a  monotone  Increasing  sequence  of  approximations  to  the  solution 

of  the  differential  equation 

(1)  —  -  $(u,t) »  u(o)  -  c, 

dt 

provided  that  we  assume  that  4(u,t)  Is  a  twice  differentiable 
convex  function  of  u  In  some  t-interval  [o,tQ].  Similarly, 
monotone  decreasing  sequences  can  be  obtained  if  4  1b  concave. 
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A  NOTE  ON  MONOTONE  CONVERGENCE  TO  SOLUTIONS  OP 
PIRST  ORDER  DIPPERENTIAL  EQUATIONS 

By 

Richard  Bellman 

jl .  Introduction . 

The  purpoee  of  this  brief  note  is  to  show  that  we  can  obtain 

a  monotone  Increasing  sequence  of  approximations  to  the  solution 

of  the  differential  equation. 

(1)  —  -  <*(u,t),  u(o)  -  c, 

dt 

provided  that  we  assume  that  <>(u,t)  Is  a  twice  differentiable 
convex  function  of  u  In  some  t-lnterval  Lo,t0].  Similarly, 
monotone  decreasing  sequences  can  be  obtained  If  4  Is  concave. 

The  connection  between  the  methods  and  results  discussed 
here  and  the  theory  of  dynamic  programming  Is  treated  in  *1] . 

§2.  The  Rlccatl  Equation . 

Let  us  begin  by  considering  a  particularly  important  case, 

the  Rlccatl  equation, 

(1)  —  -  u2  ♦  a(t) ,  u ( 0 )  -  c. 

dt 

Assume  that  a(t)  Is  lntegrable  over  some  Initial  Interval 

o  £  t  £  te. 

The  basic  relation  we  shall  employ  is 

2  o 

u  ■  Max  (2uv  -  v‘  ) . 

v 


(2) 
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We  can  then  write  (1)  In  the  form 

(3)  —  »  Max  [  2uv-v2  ♦  a(t)  ],  u(o)  -  c, 

dt  v 

where  v  ranges  over  all  functions  defined  over  [o,tQj. 

Ab  pointed  out  In  [l]  ,  (3)  leads  to  the  result 

(4  )  u  -  Max  IT  (v , t ) , 

v 

where  IT  la  the  solution  of 

(5)  3T  .  JTJV-v?  +  a(t ) ,  TTXo  J  -  c, 
dt 

for  a  fixed  function  v  -  v(t). 

Let  u s  now  uee  the  representation  In  (3)  to  obtain  a  monotone 

Increasing  sequence  of  approximations  to  the  solution  of  (1). 

Let  vQ(t)  be  an  lntegrable  function  over  [o,tQ],  and  let  uQ  be 

defined  as  the  solution  of 

du  P 

(6)  -  -  2  uqvq  -  vQd  +  a ( t ) ,  uQ(o)  -  c. 

d  t 

Let  Vj(t)  be  the  function  which  maximizes  the  quantity 
(7  )  g(v)  -  2  uqv  -  v2  a ( t ) , 

which  is  to  say  *  uQ.  Now  define  the  next  approximation  u^ 
as  the  solution  of 

du  1  2 

-  -  2  u^v^  -  v^  +  a(t),  u^(o)  ■  c. 

d  t 

In  this  way  we  obtain  the  recurrence  relations 
du  ,  ? 

(c)  -7—  -  2  Un  +  1  un  -  %  +  »(t),  un+1(°)  -  c, 

dt 

r>-0,l,2,...,  which  we  recognize  as  Newton's  method  applied  to 
the  original  equation  In  (l). 
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Let  ub  note  in  passing  that  the  method  we  have  employed 
to  derive  this  sequence  of  approximations  Is  a  particular 
application  of  the  concept  of  "approximation  In  policy  space", 
cf.  [2]. 

We  wish  to  show  tnat 

(9)  uo  <  ux  £  ...  £  un  ^  ...» 

for  o  £  t  £  t^ ,  where  [o,^]  Is  some  common  Interval  of  defini¬ 
tion. 

It  Is  easy  to  show  using  standard  techniques  that  we  can 
ensure  a  common  Interval.  Let  us  consequently  turn  to  the 
proof  of  monotonlclty .  We  have 

du  p  p 

(10)  TT  ■  2  Wl  -  u  n-l  *  *('l  1  !  Vn  -  “n 
dt 

p 

since  v  ■  un  maximizes  2  uRv  -  v  4  a(t).  Comparing  this 
last  equation  with  (8),  we  see  that  we  will  have  established  the 
desired  monotonlclty  If  we  can  prove 

Lemma:  Let  x(t)  be  a  function  satisfying  the  Inequality 

■  c ,  t  >  0 , 


«  c,  t  >  0, 


then 

(13) 


(11)  —  £  p(t)x  4  q(t),  x  jo) 
dt 

and  y(t)  satisfy  the  equality 

(12)  ^  -  p(t)y  4  q (t ) ,  y(o) 

dt 


x(t)  £  y(t), 


Proof:  The  relation  In  (11)  la  equivalent  to 
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(14)  —  -  p  ( t )  x  *■  q  (t )  -  r(t),  x  (o )  - 
dt 

with  r(t)  >  o.  Solving  for  x  we  see  that 

t  p(z)dt 

(15)  x  -  y  -  ^r(a)e  da, 

o 

which  deraonatratea  (13)* 


c, 


§3.  The  General  One- Dimensional  Case . 

Consider  the  equation 

(1)  —  -  <*(u,t),  u(o)  -  c. 
dt 

Let  ub  now  prove 

Theorem  1 .  The  sequence  of  Approximations  £  u^  j defined  by 
du 

(2)  -  -  4(vQ,t)  +  (u0-vQ)  ^’(v0,t),  uQ(o)  -  c, 

dt 


du 


n+1 


dt 


<>(un’t)  +  (unH-u»)  ur»l(o)  "  0 


( ’  denotes  the  partial  derivative  wl th  reapect  to  u) 

Is  monotone  Increasing,  uQ  <  u^  <  . . .  <  ur  <  . . . ,  within  a  common 
Interval  [o,tQj,  provided  that  vQ(t)  _ls  an  lntegrable  function 
over  some  Interval  [o,t^j,  and  that  the  function  q(u,t)  poaaeaaea 
a  non-negative  second  derivative  In  u  for  o  <  t  <  t ^ . 

Proof:  The  first  point  to  observe  Is  that  the  generalization 
of  {2.2)  le 

(3)  q(u,t)  -  Max  £  4(v,t)  ♦  (u-v)  4'  (v,t)  j  . 
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The  existence  of  a  common  interval  of  existence  follows  standard 
lines,  and  the  remainder  of  tne  proof  follows  the  lines  of  }?. 

54 .  jjultl- Dimensional  Case . 

If  f(x,y,t)  and  g(x,y,t)  are  convex  functions  of  x  and  y 
within  some  t— Interval,  it  is  easy  to  see  that  we  may  write  the 
system  of  equations 


—  -  f (x ,y , t ) ,  x ( 0 )  -  a. 


-  g(x,y,t),  y(o)  -  b, 


In  the  form 


(2)  -  Ma*  rf(u,v,t)  ♦  (x-u)  2L  +  (y-v)  2L  *,  x(o)  -  a, 

dt  u,v  L  au  fiv  J 

&  m  Max  I"  g(u,v,t)  ♦  (x-u)  ££  +  (y-v)  ££  ],  y(o)  -  b, 

dt  u,v  L  (9u  (?v  J 

The  analogue  of  the  Lemma  in  §2  is  not  unrestrictedly  true, 
however.  If  x  and  y  satisfy  the  inequalities 

(5)  ^  an(t)x  4-  a12(t)y  ♦  r1(t),  x(o)  -  c, 

dt 

^  1  a21(t)x  ♦  a22(t)y  ♦  r2(t),  y ( 0 )  -  c, 


It  Is  not  always  true  that  x  and  y  are  respectively  bounded  by 
the  solutions  of  equalities. 

For  the  case  where  the  a^j  are  constants,  a  necessary  and 
sufficient  condition  that  this  be  so  is  that  >0,  i  4  J*  and 
for  the  general  case  a  simple  sufficient  condition  is  a ^ ^ ( t )  £  0, 
with  a  necessary  condition  of  quite  complicated  and  unusable  type, 
Of.  [5],  p.  14. 
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